Technical Field
The present disclosure relates to technology for acquiring an image of a subject using a moving camera.
Description of the Related Art
In order to continuously shoot a subject, it is necessary to have a camera line of sight trained on a subject (physical object). This task is often undertaken manually by the photographer, but it is difficult to perfectly track something like the bouncing of a ball that is high speed and has irregular movement. For this reason, research into systems for automatically controlling the line of sight direction of a camera mechanically (so-called Active Vision: refer to Non-patent Publication 1 below) has become widespread in many fields.
With normal Active Vision technology, since the camera itself is moved while being attached to a drive platform, there is a delay in response speed with respect to movement in the line of sight direction. This makes tracking of a moving object that includes sudden acceleration changes (for example, a ball being used in a ball game) difficult. If the fact that the frame rate of a high speed camera reaches 1,000,000 fps in faster applications, and actual conditions where image processing is carried out at high speed by a GPU, are considered, it can be said that the line of sight control speed is a bottleneck to the speed aspects in various tracking systems.
In order to solve this problem, an optical system known as a Saccade Mirror has been proposed, to carry out change of the line of sight of a camera at high speed using small drive mirrors arranged in front of a camera (refer to Non-patent Publication 2 below). With this technology, using two-axis galvano mirrors makes high-speed line of sight change possible. Regarding a control system, if it were possible to control the line of sight so as to always keep a physical object in the center of the screen, it can be considered that unprecedented dynamic shooting would become possible.
However, in order to inspect structures that have been placed on a road, if sharp images of the structures can be obtained from a camera that is placed on a vehicle that is moving at high speed, it can be considered that it will be possible to simplify inspection operations. However, a system structure of a conventional Saccade Mirror is merely for capturing an image while tracking a physical object that moves at high speed. That is, a system of a conventional Saccade Mirror is not for handling a case where a camera itself is moving at high speed with respect to the subject. Also, with a conventional Saccade Mirror, when a physical object moves away from the center of the screen, control is carried out to correct mirror angle so that the physical object is repositioned at the center of the screen. If this is done, motion blur will occur within a captured image, accompanying adjustment of mirror angle. Under such circumstances it is considered difficult to use a conventional Saccade Mirror to handle inspections that requires high resolution.
In a situation where a physical object that is fixed is captured while moving a camera at high speed (for example 100 km/h), motion blur arises in the captured image. In order to accurately inspect a damaged state of the physical object using an image it is necessary to either lower the movement speed of a vehicle (namely movement speed of the camera), or shorten the exposure time. If movement speed of the vehicle is lowered, it is likely to result in deterioration in inspection efficiency. Also, if the exposure time is shortened it is likely to result in lowering of image quality.
There have been various proposals for technology to correct so-called handshake. However, that has been no technology proposed to alleviate the effects of motion blur in an image that has been acquired by a camera that moves at high speed.
Technology for alleviating motion blur by applying image processing (so-called deconvolution) to an image after capture has also been proposed. This image processing can be mainly divided into deconvolution, where PSF is known, and blind deconvolution, where PSF is not known. However, both types of image processing require calculation time, and so there is a problem in that real-time processing is difficult. For this reason, this procedure is not suitable for tracking control such as tracking a physical object using image information. Also, since this image processing is correction processing after motion blur has occurred, there is a problem in that it is basically not possible to restore to an original in the event that image information having high spatial frequencies has been lost.
Also, a procedure for reducing motion blur by applying pan/tilt to a camera itself in a direction in which a physical object moves, using actuators, has been proposed in Patent Publication 1 below. However, with this technology since the camera itself is driven the weight of the camera is a constraint making high-speed line of sight control difficult. For example, it is considered that an operation such as changing the line of sight direction from a camera, that moves at high speed, at a higher speed as movement speed of the camera itself becomes negligible would be difficult.
Also, the technology in Non-patent Publication 2 below is technology for continuously capturing images while tracking a physical object that moves at high speed, and is not intended to capture a physical object while a camera is moving at high speed.